Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(storage): retry broken pipe error #10374

Merged
merged 1 commit into from
Jun 12, 2024
Merged

Conversation

tritone
Copy link
Contributor

@tritone tritone commented Jun 12, 2024

This is another retryable net.OpError that can arise when a connection has been closed. So far we've only seen this in TestRetryConformance cases rather than in customer issues, but there is no reason not to do a retry and we think it can cut down on flakes from this test.

I'm going to do another PR to add this to google-api-go-client since it often occurs on uploads.

Updates #9178

This is another retryable net.OpError that can arise when a
connection has been closed. So far we've only seen this in
TestRetryConformance cases rather than in customer issues, but
there is no reason not to do a retry and we think it can cut
down on flakes from this test.

I'm going to do another PR to add this to google-api-go-client
since it often occurs on uploads.

Updates googleapis#9178
@tritone tritone requested review from a team as code owners June 12, 2024 18:39
@product-auto-label product-auto-label bot added the api: storage Issues related to the Cloud Storage API. label Jun 12, 2024
@tritone tritone merged commit 2f4daa1 into googleapis:main Jun 12, 2024
8 checks passed
@tritone tritone deleted the broken-pipe-err branch June 12, 2024 19:28
tritone added a commit to tritone/google-api-go-client that referenced this pull request Jun 12, 2024
A few fixes to shouldRetry, which is used only for storage
uploads.

* Remove attempt to unwrap syscall errors and use string matching
instead, as is used in the veneer client layer for storage.
* Use errors.Is and appropriate sentinel for net.ErrClosed.
* Add broken pipe error, which is causing flakes in veneer layer
tests (see googleapis/google-cloud-go#10374
* Add unit test for shouldRetry.

Closes googleapis/google-cloud-go#9178
tritone added a commit to googleapis/google-api-go-client that referenced this pull request Jun 12, 2024
* fix(storage): update shouldRetry for GCS uploads

A few fixes to shouldRetry, which is used only for storage
uploads.

* Remove attempt to unwrap syscall errors and use string matching
instead, as is used in the veneer client layer for storage.
* Use errors.Is and appropriate sentinel for net.ErrClosed.
* Add broken pipe error, which is causing flakes in veneer layer
tests (see googleapis/google-cloud-go#10374
* Add unit test for shouldRetry.

Closes googleapis/google-cloud-go#9178

* don't use xerrors
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants